Assessing Quality of Product Reviews

In the past few years, there has been an increasing interest in mining opinions from product reviews [3][4][5]. However, due to the lack of editorial and quality control, reviews on products vary greatly in quality. Thus, it is crucial to have a mechanism capable of assessing the quality of reviews and detecting low-quality and noisy reviews. Some shopping sites already provide a function of assessing the quality of reviews. For example, Amazon 2 allows users to vote for the helpfulness of each review and then ranks the reviews based on accumulated votes. However, according to our survey, users’ votes at Amazon have three kinds of biases as follow: 1) Imbalance vote bias -users tend to value others’ opinions positively rather than negatively. As the result, most reviews are considered as high-quality ones; 2) Winner circle bias -the more votes a review gains, the more default authority it would appear to readers, which in turn will influence the objectivity of the readers’ votes. As the result, a few reviews receive most of users’ votes; 3) Early bird bias -the earlier a review is posted, the more votes it will get. Therefore, some high-quality reviews may get fewer users’ vote because of later publication. Existing studies [2][6] used these users’ votes for training ranking models to assess the quality of reviews, which therefore are subject to these biases. In our research, we identify the aforementioned biases and define a standard specification to measure the quality of product reviews. We then manually annotate a set of ground-truth with real world product review data conforming to the specification.